Mixture of normals with strong separation
Setup
The general form of the model is:
\[ \begin{align} y_{gi} &\sim N(\mu_{gi}, \sigma_g^2), \\ \mu_{gi} &= \beta_{g0} + x_i(\beta_{g1}D_g), \end{align} \] for \(g = 1, \dots, G\) and \(i = 1, \dots, N_g\).
In all cases, true values for the regression parameters are drawn from: \[ \begin{align} \beta_{g0} &\sim N(0, 1) ~\text{and}~\\ \beta_{g1} &\sim 0.5 N(-M, 1) + 0.5 N(M, 1). \end{align} \] To visualize the simulating distribution for \(\beta_{g1}\):
\(G = 50, N_g = 10\) and \(x_i = 0, i \leq 5\) and \(1\) otherwise.
- Model 1: \(M = 10\), \(D_g = 1\) and \(\sigma_g^2 = \sigma^2 = 0.1, ~ \forall ~ g\)
- Model 2: \(M = 4\)
- Model : \(M = 4\), \(D_g \sim bern(0.5)\) and \(\sigma_g^2 = \sigma^2 \ = 0.1, ~ \forall ~ g\)
- Model : \(D_g \sim bern(0.5)\) and \(\sigma_g^2 = \sigma^2, ~ \forall ~ g\), and \(\sigma^2\) is unknown (uniform prior)
- Model : \(D_g \sim bern(0.5)\) and \(\sigma_g^2 \sim \text{half-}t(\dots), ~ \forall ~ g\)
Model 1
The only thing this model has to do is estimate the \(\beta_{g0}\) and \(\beta_{g1}\) parameters, since the variance is known and \(D_g = 1\) (i.e. no classification problem). M = 10
Model 2
M = 4
Model 3
Introduces the two-class component of the problem. M = 10
Model 4
Introduces the two-class component of the problem. M = 4
Model 5
Introduces variance estimation, i.e. estimation of \(\sigma^2\) instead of assuming the correct value
Model 6
Introduces estimation of variance parameters for \(\beta_{g0}\) and \(\beta_{g1}\), instead of assuming both are 1
Model 7
Adds paired random effects
Model 8
Switches to Poisson response